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ABSTRACT 



A face recognition system is provided comprising an input 
process or circuit, such as a video camera for generating an 
image of a person. A face detector process or circuit deter- J 
mines if a face is present in a image. A face position) 
registration process or circuit determines a position of the ! 
face in the image if the face detector process or circuit 
determines that the face is present. A feature extractor 
process or circuit is provided for extracting at least two 
facial features from the face. A voting process or circuit 
compares the extractor facial features with a database of 
extracted facial features to identify the face. 

90 Claims, 12 Drawing Sheets 
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FACE RECOGNITION APPARATUS, It is an additional object of the present invention to 

METHOD, SYSTEM AND COMPUTER provide a method, system, apparatus and computer readable 

READABLE MEDIUM THEREOF medium for which extracts at least two facial features and 

then selects one of the features for use in the identification. 

Applicant claims priority under 35 U.S.C. § 119 (e) to 5 m ^™ T 

prov^ional application Ser No. 60/066,282, filed on Nov. SUMMARY OF THE INVENTION 

14, 1997, entitled Face Recognition Apparatus, Method, According to this invention, a face recognition system is 

System and Computer Readable Medium Thereof. provided comprising an input process or circuit, such as a 

BACKGROUND OF THE INVENTION video camera for generating an image of a person. A face 

10 detector process or circuit determines if a face is present in 

1. Field of the Invention a image. A face position registration process or circuit 
This invention relates generally to a method and system determines a position of the face in the image if the face 

for machine recognition of an image . More particularly, this detector process or circuit determines that the face is present, 
invention relates to an apparatus, a method, a system and a A feature extractor process or circuit is provided for extract- 
computer readable medium for automatic recognition a 15 ing at least two facial features from the face. A voting 
human face and then an identification of the recognized face process or circuit compares the extractor facial features with 
from a database of previously stored face data. a database of extracted facial features to identify the face. 

2. Description of the Related Art According to another aspect of the present invention, the 
Machine vision has many commercial applications and 2Q face detector is provided with a lumen normalization circuit 

therefore, has attracted much attention in recent years. Many for lumen normalizing the output by the input circuit. An 

machine vision techniques and methods have been devel- edge extractor extracts edge information from an output of 

oped for detecting various image patterns and objects the lumen normalization circuit; A down sampling circuit is 

including deformable objects such as human faces. The provided for down sampling at a predetermined ratio an 

ability to recognize a human face is an important machine 25 output signal from the edge extraction circuit. A facial 

vision problem. Face recognition applications are numerous, template circuit for applies a plurality of facial templates to 

as well as diverse. an output of the down sampling circuit. A facial template 

Face recognition applications can be used, for example, matching circuit for determines which one of the plurality of 

by security agencies, law enforcement agencies, the airline facial templates corresponds to the output of the down 

industry, the border patrol, the banking and securities indus- 30 sampling circuit. 

tries and the like. Examples of potential applications include According to an additional aspect of the present 

but are not limited to entry control to limited access areas, invention, the face position registration circuit is provided 

access to computer equipment, access to automatic teller with a lumen normalization circuit for lumen normalizing an 

terminals, identification of individuals and the like. output by the face detector circuit. An edge extractor extracts 

Present face recognition methods used in person identi- 35 edge information from an output of the lumen normalization 

fication systems, typically employ a face recognizer which circuit, and a down sampling circuit down samples at a first 

determines the identity of a human face image. In order to predetermined ratio an output signal from the edge extrac- 

identify the image, the system compares the face to other tion circuit. An eye template circuit applies a plurality of eye 

faces stored in a database. Systems used in many visual templates to an output of the down sampling circuit, and an 

monitoring and surveillance incorporate a process whereby 40 eve template matching circuit determines which one of the 

the system determines the position of the human eyes from plurality of eye templates corresponds to the output of the 

an image or an image sequence containing the human face. down sampling circuit. A face alignment circuit aligns the 

Once the position of the eyes is determined, all of other face of the image output by the input unit in accordance with 

important facial features, such as the position of the nose and the eye matching circuit. 

the mouth, are determined by methods that (1) use correla- 45 According to a further aspect of the present invention, the 

tion templates, (2) spatial image invariants, (3) view-based feature extractor is provided with a cropping circuit for 

eigen spaces, etc. The use of one of these methods enables cropping at least the first facial feature and the second facial 

the system to recognize a face from a given face database, feature output by the face position registration circuit. A 

and can also be used to perform a variety of other related lumen normalization circuit is provided for lumen normal- 

tasks. 50 izing the first facial feature and the second facial feature 

One problem associated with conventional recognition output by the cropping circuit An edge extractor extracts 

techniques is somewhat inaccurate identification of the edge information from the first facial feature and the second 

human face under analysis. In other words, the face is either facial feature output by the lumen normalization circuit. A 

misidentified or not identified at all. Moreover, some of down sampling circuit down samples at a first predetermined 

these techniques require significant computation time to 55 ratio the first facial feature and at a second predetermined 

recognize and identify a face. The conventional solution to ratio the second facial feature output signal by the edge 

this problem has been accepting slow recognition time or extraction circuit. A feature vector calculating circuit for 

using more power and therefore expensive computation calculating a first feature vector of the first facial feature 

equipment. from the down sampling circuit and a second feature vector 

en of the second facial feature output from said down sampling 

OBJECTS OF THE INVENTION circuit 

Therefore, it is an object of the present invention to According to a still further aspect of the present invention, 

overcome the aforementioned problems. the voting circuit comprises a first recognizer circuit for 

It is a further object of the present invention to provide a comparing the first feature vector with corresponding first 

method, system, apparatus and computer readable medium 65 feature vectors of a database of candidates. A second rec- 

for accurately recognizing and identifying a human face ognizer circuit for compares the second feature vector with 

from a face database. corresponding second feature vectors of the database of 
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candidates, and a selection circuit of selects one of the accomplish specific tasks. Although a particular computer 

candidates in the database in accordance with one of the first may only have some of the units illustrated in FIG. 12 or 

recognizer circuit and the second recognizer circuit. may have additional components not shown, most comput- 

Other objects and attainments together with a fuller ers will mclude at least the unite shown^ 
understanding of the invention will become apparent and 5 Specifically, computer 100 shown in FIG. 12 includes a 

appreciated by referring to the following description and random access memory (RAM) 106 for temporary storage of 

f\ . . 4 . ..u -j information such as instructions and data, a read only 

claims taken in conjunction with the accompanying draw- mcmory 1M for of ^ comput '_ 

m & s " er's configuration and basic operating commands and an 

BRIEF DESCRIPTION OF THE DRAWINGS 10 ^put/output (I/O) adapter 110 for connecting peripheral 

devices such as a disk unit 113 and printer 114 to the bus 

In the drawings wherein like reference symbols refer to 108, via cables 115 and 112, respectively. Disk unit 113, 

like parts: among other functions, stores the face recognition applica- 

FIG. 1 is a schematic representation of a preferred tion software and a database of face information for pro- 
embodiment in accordance with the present invention; 15 cessing by CPU 102. Image input device 132 is also con- 

FIG. lAis a schematic representation of the face detector Dected i0 bus f 108 ™ ^»for acquiring an image 

system of FIG 1- containing a face. Image input device 132 can convert an 

n „ ' ' , c ^ c ^ , image to data usable by CPU 102. Image input device 132 

FIG^JsaflowchartofthepreferredembodimentofFIG. mfty bc by a ^ a digital still 

1' 20 camera, a flat bed scanner and the like. Alternatively, the 

FIG. 3 is a block diagram of the face recognition process image input device may be replaced by a communication 

or circuit of the preferred embodiment of FIG. 1; channel to another computer or computer network having 

FIG. 4 is a block diagram of the down sampling process previously stored images thereon. A user interface adapter 

or circuit of the preferred embodiment of FIG. 1; 116 is also provided for connecting input devices, such as a 

FIG. 5 is a block diagram of the face location registration 2 s keyboard 120, and other known interface devices including 

process or circuit of the preferred embodiment of FIG. 1; mice, speakers and microphones to the bus 108. Visual 

FIGS. 6A and 6B as shown together in FIG. 6 are flow ° ut P ut * £™ de ?. b * a f&y adapter 118, which connects 

charts of the face location registration circuit of FIG. 5; * c bus ™* * ! * e ™f 122 such f. a Vld *° ™ h 

„ « . Computer 100 has resident thereon, and is controlled and 

FIG. 7A is a block dia^am of the feature extracUon coordinated b m operating system, 
process or circuit of the preferred embodiment of FIG. 1; nG u fa niustration of computer m which 

FIG. 7B is a flow chart of the feature extractor of the mc i u des system memory or RAM 106, CPU 102 and camera 

preferred embodiment of FIG. 1; or input device 132 interconnected via a system bus 

FIG. 8 is a exemplary diagram illustrating the method of 108 well known in the computer arts. Also interconnected to 

generating a feature vector; 35 the system bus 108 is system-addressable storage device 

FIG. 9 is a block diagram of the voting process or circuit 110A and data storage interface HOB capable of accepting, 

of the preferred embodiment of FIG. 1; reading and writing information to a type of removable 

FIG. 10 is a flow chart of a routine for generating face data *edia mA external store 113B as representative stor- 

in the face recognition server of the preferred embodiment a S e mediums in communication with representative process- 

of FIG 1* 40 ing system 100. Accordingly, in this representative process- 

™^ . 4 • * •« . ing system, programming instructions corresponding to the 

FIG. 11 illustrates in more detail a representative process- . B J f f. ? , . nr -5 ^ n 

r r face recognition processing may be partially or fully con- 
ing sy em, tained within external store 113A, removable media 113B, 
FIG. 12 is a block diagram of a computer system, for or system memory 106 or ROM 104 as is well known in the 
example, a personal computer system on which a face ^ art 

recognition application program in accordance with the 4 Moreover> buffer 2 23 and buffer 210 may be constructed 
present invention can operate; and &y ^ m memory 1Q6 as ^ mtegralpart thereof or may 

FIG. 13 is a schematic diagram of distribute processing comprise discrete circuitry as is well-known in the comput- 

implementation of the present invention. m g ar ts. 

DESCRIPTION OF THE PREFERRED 50 *^^r™*t * 3A m * * ho ^ M d f' 

cwTir^T^rjxrrc CD-ROM, ROM cartridge or other apparatus suitable for 
EMBODIMENTS - * * . j ui c c- -t i 

holding information in computer readable form. Similarly, 

The present invention is preferably practiced in the con- external store 113B may include another processing system, 
text of an application program performed by a computer a computer readable storage component, a collection or 
such as the x86 compatible computer (a.k.a. IBM 55 network of processing systems and/or storage components, 
compatible), Apple Macintosh computer, Sun engineering or similar device or devices in communication with pro- 
work station and the like. The present invention consists of cessing system 2000 to exchange information including the 
primarily three major functions, namely image acquisiaton, above-mentioned thread instructions. Further, in this 
face database processing and face recognition processing, as embodiment, processing system 100 is indicated as being a 
will be explained in detail herein below. go general purpose personal computer. However, one of ordi- 

A representative hardware environment is depicted in nary skill with knowledge of the presently preferred embodi- 

FIG. 12, which illustrates a typical hardware configuration ment according to the present invention should know that 

of a computer 100 in accordance with the subject invention. the particular processing system could alternatively include, 

The computer or processing system 100 is controlled by a for example, a special-purpose dedicated micro-controlled 

central processing unit (CPU) 102, which may be at least 65 subsystem or similar processing device as long as it has 

one conventional microprocessor; a number of other units, sufficient resources for at least execution of the techniques 

all interconnected via a system bus 108, are provided to described and charted hereinbelow and has sufficient inter- 
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facing resources to communicate and exchange information cation of this system. For example, upon identification, 

with the image input member or camera 10. In this system, subject 1 could be allowed access into a building or a 

image acquisition, face recognition processing and database computer. 

processing are performed on a single computer. Face processor system 30 can also provide communica- 

In another approach, as shown in FIG. 13, a distributed 5 tion and control with interactive panel 20 and camera 10. 

processing model may be implemented. In this approach, Specifically, as noted above, subject 1, can interact with 

various components of the face recognition system are interactive panel 20 to initiate image acquisition. Face 

processed on different computers 100A, 100B, 100C . . . P roccssor W*** 30 ! u P° n rcc f f of aa ^ atl0n S1 g° al 

£ /mmlt , • * %i. u a. & om panel 20, provides a signal to camera 10 to acquire an 

100N. These computers communicate with each other r r Jr t ~ . ., n 

, 11 i *_ i ■ * a u —m image. Face processor system 30 could also provide excep- 

through a well know networking system. As can be appre- 10 & . . . r . . ^ . , , r ,. r . . J 

. f, - ,. t*ii *l. ^ . tion signals to interactive panel, for example, notifying the 

ciated by one of ordinary skill in the art, these computers T. i- u* iru * . * i .i_ 

r, t . / . * i i subject to realign himself before camera 10 to take another 

may reside at the same location or at different locations. In . J A , t .7 - . fl . . 

, r , . - . 4 . , image. A detailed explanation of the face processor system 

Ibis approach, for example, image acquisiUon may be pro- 30 ^ provided ^ ereinbelow . 
cessed by computer 100A, face recognition may be per- _ r . , . . , 

formed on computer 100B and a database processing may be is Face processor system 30 interfaces with face recognition 

performed on computer 100B. ^ f Face , ^>SP^n server 40 stores a database 

_ - , , 4 , 4 ... . . , containing facial characteristics of potential candidates for 

Details of each of these components will be presented ^k;„„ e,^ «,™„;t™ ac\ *u a f„„»™.. 

. r matching. Face recognition server 40 compares the features 

erein ow. extracted by face processor system 30 and finds, if possible, 

Reference is now made to FIGS. 1 and 1A which show the ^ statistically a matching face. Of course, if the database is of 

general configuration of the face recognition system m thc appropriate size, the database may be stored in the 

accordance with the preferred embodiment of the present processor systC m 30 instead of the separate face recognition 

invention. As will be appreciated by one of ordinary skill in server 40 

the art, the functional blocks shown in these and other p]G ^ d co^g^on of fac6 proccS sor \ 

figures are divided arbitrarily in this specificat.on for con- m 30 and nG . 2 is a flow chart of the method 

vemence of descnption only. Other arrangements are also by ^ system . Face processor system 30 | 

possi e. comprises a face dejficXQX- 31 for determining if the image 

FIG. 1 illustrates, a subject 1 positioned in the view of jiom^mera^O fstep^^ 
camera 10 for acquiring an image thereof. Camera 10 may 203]Tlf no face imagers present (step 205), face processor 
also be implemented by a multi-frame video camera capable 3Q svstem 30 performs exception handling, by for example, 
of capturing multiple, sequential images over a period of displaying a message on interactive panel 20. This message 
time. Alternatively, a scanner (not shown), other suitable may mc hide instructions for subject 1 to reposition himself 
video input device or digitizer may be substituted for camera for ^j^g me picture by camera 10. Once face detector 31 
10 to perform face recognition from a previously recorded determines that a face is present in the image, face position 
image, such as a photograph. In another embodiment, a 3S registrator 33, determin es the location of the_ face in the 
video tape recorder can be employed as a video input device ( step 207Y Face"oc^5o^eastrator 33. also dete r- 
to input video images recorded by a video camera used by, mines if t he face is properly ali gnedTjJLtl^e^ 
^for example, a surveillance system. These devices may misaUgnmentTtnelmage^ realigned, by for example rota- 
provide analog or digital image signals. Of course, analog tion Misalignment of the face may occur if subject 1 is not 
I image signals need to be converted by an analog to digital ^ pos i t j on perpendicular to camera 10 or if bis head tilted. If 
converter using conventional techniques. Such an image is me face ^ QOt be detected or is significantly out of 
[ represent by MxN pixels and is stored in memory of the alignment such that the face could not be properly registered 
system. (step 209), face position registrator 33 initiates the exception 
After subject 1 is appropriately positioned, subject 1 may handling process (step 217). Again, a message can be sent to 
manually initiate camera 10 to acquire an image or take a 45 interactive panel 20 to display a message with instructions 
picture thereof by using interactive panel 20. Alternatively, f or subject 1 to align his head for retaking the picture by 
the face recognition system may comprise a proximity camera 10. 

sensor (not shown) for automatically acquiring an image 0 nce face position registrator 33 determines the location 

when subject 1 is properly positioned in the field of vision 0 f the face, facial features of the face are extracted by feature 

of camera 10. Interactive panel 20 may consist of a display 50 extractor 35 (step 211). Feature extractor 35 extracts a 

panel (such as a CRT or LCD display) and keyboard either variety of facial features by generating feature vectors, each 

an actual keyboard or softkeys located on the display panel. representing a respective feature. In the preferred embodi- 

Interactive panel 20 may also display the image taken by ment me features extracted include feature 1—an eye, 

camera 10. In simpler applications, interactive panel 20 may feature 2— nose and both eyes, and feature— 3 mouth, nose 

consist of lights, sound making apparatuses and/or switches. 55 an d both eyes. Of course other facial features and combi- 

The output of camera 10 is provided to face processor nations may be used. Feature extractor circuit 35 assigns a 

system 30 for recognition and identification. Face processor confidence level to each of the extracted features using a 

system 30 may be implemented by programming a general neural network or statistical analysis approach. In the pre- 

purpose computer described above with appropriate appli- ferred embodiment a Probabilistic Decision-Based Neural 

cation software stored in a computer readable medium, so Network (PDBNN) is implemented. Such a PDBNN is 

discrete components, application specific integrated circuits described in, for example, "Face Recognition/Detection by 

and the like or any combination thereof. Probabilistic Decision-Based Neural Network", IEEE 

Face processor system 30 extracts a set of features from Transaction on Neural Networks, Vol. 8, No. 1, pages 

the image output by camera 10 and compares these features 116-132, January 1997, the entire contents of which are 

with a database of features stored in face recognition server 65 incorporated herein by reference. The confidence level indi- 

40. Upon identification of the face, face processor system 30 cates the degree of degree of certainty that feature extractor 

can perform a variety of activities depending on the appli- 35 extracted the appropriate feature. In the preferred 



03/31/2004, EAST Version: 1.4.1 



6,K 

7 

embodiment, three confidence levels are assigned, one for 
each of features 1, 2 and 3, respectively. Face recognition 
server 40 stores the same number of features extracted by 
feature extractor 35 for each potential candidate. In the 
preferred embodiment, face recognition server 40 stores 
three facial features for each potential candidate. A detailed 
description will be provided hereinbelow with respect to 
FIG. 10 on how to develop the database. Voting circuit 37 
provides each of the feature vectors extracted by feature 
extractor 35 to face recognition server 40 (step 213). Face 
recognition server 40 selects the stored feature vectors with 
the highest level of confidence based statistical analysis 
techniques (step 215). Of course in an alternate embodiment, 
the face recognition server may be part of face processor 
system 30 instead of as a separate processor. 

There are a variety of ways the result the identification 
may be utilized by output device 38 depending on the 
application of the face recognition system. For example in a 
simple application, if an authorized subject is recognized, a 
door can be unlock to allow the person access to a building. 
Alternatively, if an authorized subject is recognized, that 
subject is allowed access to a computer. This system can 
replace or supplement conventional password schemes. In 
another embodiment, a recognized subject can be authorized 
to purchase goods and/or services at an actual store or on the 
Internet. In a more complex application, the preferred 
embodiment can identify subjects from images taken by a 
surveillance camera and provide such identification to the 
appropriate persons. 

FIG. 3 provides a detailed explanation of face detector 31. 
As shown therein, the image acquired by camera 10 is 
processed by lumen normalizer 301 to perform lumen nor- 
malization or histogram equalization to compensate for any 
wide variation in illumination of the image of human being 
100. This circuit can increase the dynamic range of the 
image. One such lumen normalization method is discussed 
in European Patent Application 0,805,416 A2. The image is 
then processed by edge extractor 303 to determine the edge 
of the face in the image. 

The image is then processed by down sampling circuit to 
reduce the number of pixels processed without decreasing 
the effectiveness of this system while reducing the amount 
of computational power required. In general the down 
sampling reduces the number of pixels in a ratio of l:n. In 
the preferred embodiment, the down sampling ratio is 1:8. 
FIG. 4 shows in more detail the down sampling circuit. 
Initially, image data represented by MxN pixels (401) is 
filtered by a low pass filter (403) using conventional tech- 
niques. At this point the image is raster scanned, and every 
n** pixel is read and then rewritten into buffer 210 (405). As 
a result of this circuit, new data consisting of M/nxN/n 
pixels are stored in the buffer 223 (407). 

Turning back to FIG. 3, after the image is down sampled, 
predefined facial templates are applied to the down sampled 
image by facial template circuit 307. These predefined 
templates represent facial characteristics, and may be stored 
in a read only memory (ROM), main computer memory, disk 
memory and the like or any combination thereof. The facial 
templates were developed using neural network techniques 
as discussed above or statistical analysis methods. Template 
matching circuit 309 compares the facial templates to the 
downsized image and determines if a match occurs. In other 
words, template matching circuit 309 determines whether a 
face exists in the image. The position of the face is recorded 
in memory. If a face exists processing proceeds; otherwise 
the exception handling procedure is executed, as shown in 
FIG. 2. 
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Turning back to FIG. 1A and as noted above, after the 
image is processed by face detector 31, the image, in 
accordance with the position of the face stored in memory, 
is then processed by face position registrator 33. FIG. 5 

5 shows in more detail face position registrator 33. Face 
position registrator 33 comprises lumen normalizer 501, 
edge extractor 503 and down sampling circuit 505. These 
circuits are similar to those of face detector 31 and no further 
detailed discussion will be presented. 

10 After the down sampling by down sampling circuit 505, 
the image is processed by eye template circuit 507. Pre- 
defined eye templates are applied to the down sampled 
image thereof. These predefined eye templates represent eye 
characteristics, and may be stored in a read only memory 

15 (ROM), main computer memory, disk memory and the like 
or any combination thereof. The eye templates were also 
developed using neural network techniques. Template 
matching circuit 509 compares the eye templates to the 
downsized image and determines if a match occurs. If a 

20 match occurs, its position is recorded; otherwise the excep- 
tion handling procedure, as discussed above, is then 
executed. Face aligner 511 judges if the face is properly 
aligned by determining if an imaginary line connecting the 
eyes is parallel to the horizontal direction of the image. If 

25 possible the image is rotated to bring the face into alignment; 
otherwise the exception handling procedure is then 
executed. 

FIG. 6A and 6B are flow charts illustrating the operation 
of face position registrator 33. As noted above, the image is 

30 lumen normalized (step 601). A down sampling ratio is then 
selected (step 603) and the image is then down sampled in 
accordance with the selected ratio (step 605). An eye tem- 
plate for one eye, for example the right eye, is applied to the 
image (step 607) and template matching is then performed 

35 (step 609). If there is a match, that is if the confidence level 
is greater that a predetermined threshold, then processing 
proceed to the other eye. Otherwise this processing is 
repeated with the ratio of the down sampling changed by 
incrementing an index counter (step 617) to apply the next 

40 ratio. Thus steps 603, 605, 607, 609 and 611 are repeated. It 
is noted that there are S possible different down sampling 
ratios. If there is still not a match (step 609) and all S down 
sampling ratios have been used, then the exception handling 
procedure, as discussed above, is then executed. 

The operation of face position registrator 33 continues for 
detecting the other eye, by processing a mirror image of the 
eye template. In step 613 the down sampling index is reset 
and steps 621, 623, 625, 627, 629, 635, 637 and 639 are 

50 processed similarly to steps 603, 605, 607, 609, 611, 615, 
617 and 619, respectively. Thus no further discussion of 
these steps need be provided. Alternatively, an eye template 
having both eyes may be utilized instead. 
Once both eyes have been detected, face position regis- 

55 trator 33 determines if the eyes are horizontally aligned. If 
they are aligned the face position registrator 33 has finished 
processing. Otherwise, the image is rotated to bring it into 
horizontal alignment. Additionally, the location of the eyes 
are stored for later use by feature extractor 35. 

60 FIG. 7A is a block diagram of the feature extractor and 
FIG. 7B is a flow chart thereof the operation of feature 
extractor circuit 35. As noted above, in the preferred 
embodiment there are three facial features to be extracted by 
feature extractor 35. Initially, the first feature, is selected for 

65 extraction from the image (step 701). In accordance with the 
location of the face detected by face detector 31, the first 
feature, namely, both eyes, mouth and nose area of the 
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image, is cropped by cropper 750 (step 703). Lumen nor- 
malization is performed on the cropped image by lumen 
normalizer 752 (step 707) and then that image is down 
sampled by down sampling circuit 754 (step 707). The down 
sampling ratio of down sampling circuit 754 is dependent on 
the feature being extracted. The following chart shows the 
feature vs. down sampling ratio in accordance with the 
preferred embodiment: 



10 



Feature 


Down Sampling Ratio 




single eye 


1:1.1 




eyes and nose 


1:1.25 




eyes, nose and mouth 


1:15 


15 



In step 709 the x,y directional projection or feature vector 
is calculated by feature vector calculator 756. FIG. 8 illus- 
trates the calculation by feature vector calculator 756 of the 
feature vector in which an MxN image is downsize to an IxJ 
image. For each row, the values of each of the elements in 
a row are summed as shown in equation (1) to form the X ; 
components of the feature vector. 



= 1* 



(1) 



Similarly, for each column, the values of each of the 
elements in a column are summed as shown in equation (2) 
to form the Y 7 components of the feature vector. 



(2) 



The feature vector comprises the set of X components 
form 1 to I and Y components from 1 to J as shown in 
equation (3). 



(3) 



20 



25 



Steps 703-709 are repeated for each of the features by 
incrementing the feature index (step 713). 

Referring to FIG. 9, upon completion of extracting all of 
the features (step 711), the feature vectors are passed onto 
voting circuit 37. Voting processor 37 comprises one face 
recognizer (913, 915, 917, 919) for each of the features (905, 
907, 909 and 910). An inquiry is made of face recognition 
server 40 to find the closest set of feature vectors. In this 
circuit a confidence level for each of the features vectors 
(905, 907, 909 and 910) with relation to the closest stored 
feature vectors is determined. The voting sub circuit 901 
selects the stored set of feature vectors based on the highest 
confidence level. This stored set of feature vectors represents 
the person having the recognized face from the database of 
candidates and the face is identified (903). Output process- 
ing is performed by output 38 as described above. 

FIG. 10 illustrates a process in accordance the present 
invention for creating and storing feature vectors in the face 
recognition server of each candidate. Initially an image of a 
subject is generated similarly as described above (step 
1002). In step 1003, the features of the subject's face are 
generated similarly to the process described above by the 
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face detector 31, face position registrator 33 and feature 
extractor 35. Since these circuits are quite similar, no 
additional explanation is provided here. The image is gen- 
erated Q times (step 1001) for slightly varied poses by the 
subject, and real feature vectors for each pose are then 
generated. A set of virtual feature vectors is then generated 
based on computer generated variations of the real feature 
vectors (steps 1005 and 1007). An average is taken of the 
real feature vectors and the virtual feature vectors (step 
1009). This result is then stored in face recognition server 40 
(step 1011) to represent one candidate. This process is 
repeated for each candidate. 

As will be appreciated by one of ordinary skill in the art, 
after obtaining a fuller understanding of the present 
invention, may recognize other types of objects or other 
features of subject 10 using similar techniques as discussed 
herein. 

While the invention has been described in conjunction 
with several specific embodiments, it is evident to those 
skilled in the art that many further alternatives, modifica- 
tions and variations will be apparent in light of the foregoing 
description. Thus, the invention described herein is intended 
to embrace all such alternatives, modifications, applications 
and variations as may fall within the spirit and scope of the 
appended claims. 

What is claimed is: 

1. A face recognition system comprising: 

an input unit for inputting an image of a subject; 

a face detector for determining if a face is present in the 
image from said input circuit; 

a face position registration circuit for determining a 
position of the face in the image if said face detector 
determines that the face is present, said face position 
registration circuit comprising an eye template circuit 
that applies an eye template to a first side of the face 
and a mirror image of the eye template to a second side 
of the face; 

a feature extractor for extracting at least first and second 
facial features from the face in accordance with the 
position determined by said face position registration 
circuit; and 

a voting circuit for comparing the at least two extracted 
facial features with a database of candidates corre- 
sponding facial features to identify the face. 

2. A face recognition system according to claim 1, 
wherein said face detector comprises: 

a lumen normalization circuit for lumen normalizing the 
output by said input circuit; 

an edge extractor for extracting edge information from an 
output of said lumen normalization circuit; 

a down sampling circuit for down sampling at a prede- 
termined ratio an output signal from said edge extrac- 
tion circuit; 

a facial template circuit for applying a plurality of facial 
templates to an output of said down sampling circuit; 
and 

a facial template matching circuit for determining which 
one of the plurality of facial templates corresponds to 
the output of said down sampling circuit 

3. A face recognition system according to claim 2, 
wherein the image output by said input device comprises 

MxN pixels, wherein M and N are positive integers, 
wherein said down sampling circuit comprises: 
a low pass filter to filter the MxN pixels; 
a down sampler for sampling every n tA pixel from said 

low pass filter to output M/nxN/n pixels, wherein n 

is a positive integer. 
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4. A face recognition system according to claim 1, 
wherein said face position registration circuit comprises: 

a lumen normalization circuit for lumen normalizing an 

output by said face detector circuit; 
an edge extractor for extracting edge information from an 5 

output of said lumen normalization circuit; 
a down sampling circuit for down sampling at a first 

predetermined ratio an output signal from said edge 

extraction circuit; 
an eye template circuit for applying a plurality of eye 

templates to an output of said down sampling circuit; 
an eye template matching circuit for determining which 

one of the plurality of eye templates corresponds to the 

output of said down sampling circuit; and l5 
face alignment circuit for aligning the face of the image 

output by said input unit in accordance with said eye 

matching circuit 

5. A face recognition system comprising: 

an input unit for inputting an image of a subject; 20 
a face detector for determining if a face is present in the 

image from said input circuit; 
a face position registration circuit for determining a 
position of the face in the image if said face detector 
determines that the face is present, said face position 25 
registration circuit comprising: 
a himen normalization circuit for lumen normalizing an 

output by said face detector circuit; 
an edge extractor for extracting edge information from 

an output of said lumen normalization circuit; 30 
a down sampling circuit for down sampling at a first 

predetermined ratio an output signal from said edge 

extraction circuit; 
an eye template circuit for applying a plurality of eye 

templates to an output of said down sampling circuit; 35 
an eye template matching circuit for determining which 

one of the plurality of eye templates corresponds to 

the output of said down sampling circuit; and 
a face alignment circuit for aligning the face of the 

image output by said input unit in accordance with 40 

said eye matching circuit; 
a feature extractor for extracting at least first and second 
facial features from the face in accordance with the 
position determined by said face position registration ^ 
circuit; and 

a voting circuit for comparing the at least two extracted 
facial features with a database of candidates corre- 
sponding facial features to identify the face; 

wherein said eye template circuit applies an eye template 50 
to a first side of the face and a mirror image of the eye 
template to a second side of the face. 

6. A face recognition system according to claim 4, 
wherein said eye template circuit applies an eye template 
comprising a right eye and a left eye. S5 

7. A face recognition system comprising: 

an input unit for inputting an image of a subject; 

a face detector for determining if a face is present in the 

image from said input circuit; 
a face position registration circuit for detennining a 60 

position of the face in the image if said face detector 

determines that the face is present, said face position 

registration circuit comprising: 

a rumen normalization circuit for lumen normalizing an 
output by said face detector circuit; 65 

an edge extractor for extracting edge information from 
an output of said lumen normalization circuit; 
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a down sampling circuit for down sampling at a first 

predetermined ratio an output signal from said edge 

extraction circuit; 
an eye template circuit for applying a plurality of eye 

templates to an output of said down sampling circuit; 
an eye template matching circuit for determining which 

one of the plurality of eye templates corresponds to 

the output of said down sampling circuit; and 
face alignment circuit for aligning the face of the image 

output by said input unit in accordance with said eye 

matching circuit; 
a feature extractor for extracting at least first and second 
facial features from the face in accordance with the 
position determined by said face position registration 
circuit; and 

a voting circuit for comparing the at least two extracted 
facial features with a database of candidates corre- 
sponding facial features to identify the face; 

wherein if said eye matching circuit does not determine 
that any one of the plurality of eye templates corre- 
sponds to the output of said down sampling circuit, said 
down sampling circuit down samples at a second 
predetermined ratio the output signal from said edge 
extraction circuit. 

8. A face recognition system comprising: 

an input unit for inputting an image of a subject; 

a face detector for determining if a face is present in the 

image from said input circuit; 
a face position registration circuit for determining a 

position of the face in the image if said face detector 

determines that the face is present; 
a feature extractor for extracting at least first and second 

facial features from the face in accordance with the 

position determined by said face position registration 

circuit; and 

a voting circuit for comparing the at least two extracted 
facial features with a database of candidates corre- 
sponding facial features to identify the face; 
wherein said feature extractor comprises: 

a cropping circuit for cropping at least the first facial 
feature and the second facial feature output by said 
face position registration circuit; 
a lumen normalization circuit for lumen normalizing 
the first facial feature and the second facial feature 
output by said cropping circuit; 
an edge extractor for extracting edge information from 
the first facial feature and the second facial feature 
output by said lumen normalization circuit; 
a down sampling circuit for down sampling at a first 
predetermined ratio the first facial feature and at a 
second predetermined ratio the second facial feature 
output signal by said edge extraction circuit; and 
a feature vector calculating circuit for calculating a first 
feature vector of the first facial feature from said 
down sampling circuit and a second feature vector of 
the second facial feature output from said down 
sampling circuit. 

9. A face recognition system according to claim 8, 
wherein the first predetermined ratio is greater than the 
second predeterrnined ratio. 

10. A face recognition system according to claim 8, 
wherein the first facial feature comprises a single eye. 

11. A face recognition system according to claim 8, 
wherein the second facial feature comprises both eyes and 
nose. 
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12. A face recognition system according to claim 8, 
wherein said cropping circuit crops a third facial output 

by said face position registration circuit; 

wherein said lumen normalization circuit lumen normal- 
izes the third facial feature output by said cropping 5 
circuit detector circuit; 

wherein said edge extractor extracts edge information 
from the third facial feature output by said lumen 
normalization circuit; 

wherein said down sampling circuit down samples at a 10 
third predetermined ratio the third facial feature output 
by said extraction circuit; and 

wherein said feature vector calculating circuit calculates a 
third feature vector of the third facial feature from said 
down sampling circuit. 15 

13. A face recognition system according to claim 12, 
wherein the third facial feature comprises both eyes, nose 
and mouth. 

14. A face recognition system according to claim 12, 
wherein the first predetermined ratio is greater than the 20 
second predetermined ratio, and wherein the second ratio is 
greater than the third ratio. 

15. A face recognition system according to claim 8, 
wherein said voting circuit comprises: 

a first recognizer circuit for comparing the first feature 25 
vector with corresponding first feature vectors of the 
database of candidates; 

a second recognizer circuit for comparing the second 
feature vector with corresponding second feature vec- 
tors of the database of candidates; and 30 

a selection circuit of selecting one of the candidates in the 
database in accordance with one of said first recognizer 
circuit and said second recognizer circuit. 

16. A face recognition system comprising: 

an input unit for inputting an image of a subject; 35 
a face detector for determining if a face is present in the 

image from said input circuit; 
a face position registration circuit for determining a 
position of the face in the image if said face detector ^ 
determines that the face is present; 
a feature extractor for extracting at least first and second 
facial features from the face in accordance with the 
position determined by said face position registration 
circuit; and 45 
a voting circuit for comparing the at least two extracted 
facial features with a database of candidates corre- 
sponding facial features to identify the face; 
wherein said feature extractor comprises: 

a cropping circuit for cropping at least the first facial so 
feature and the second fecial feature output by said 
face position registration circuit; 
a down sampling circuit for down sampling at a first 
predetermined ratio the first facial feature and a 
second predetermined ratio the second facial feature 55 
output signal by said cropping circuit; and 
a feature vector calculating circuit for calculating a first 
feature vector of the first facial feature from said 
down sampling circuit and a second feature vector of 
the second facial feature output from said down 60 
sampling circuit. 

17. A face recognition system according to claim 16, 
wherein the first predetermined ratio is greater than the 
second predetermined ratio. 

18. A face recognition system according to claim 16, 65 
wherein said cropping circuit crops a third facial output 

by said face position registration circuit; 
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wherein said down sampling circuit down samples at a 

third predetermined ratio the third facial feature output 

by said cropping circuit; and 
wherein said feature vector calculating circuit calculates a 

third feature vector of the third facial feature from said 

down sampling circuit. 

19. A face recognition system according to claim 18, 
wherein the first predetermined ratio is greater than the 
second predetermined ratio, and wherein the second ratio is 
greater than the third ratio. 

20. A face recognition system according to claim 18, 
wherein the third facial feature comprises both eyes, nose 
and mouth. 

21. A face recognition system according to claim 16, 
wherein said voting circuit comprises: 

a first recognizer circuit for comparing the first feature 
vector with corresponding first feature vectors of a 
database of candidates; 

a second recognizer circuit for comparing the second 
feature vector with corresponding second feature vec- 
tors of the database of candidates; and 

a selection circuit of selecting one of the candidates in the 
database in accordance with one of said first recognizer 
circuit and said second recognizer circuit. 

22. A face recognition system according to claim 16, 
wherein the first facial feature comprises a single eye. 

23. A face recognition system according to claim 16, 
wherein the second facial feature comprises both eyes and 
nose. 

24. A face recognition system according to claim 1, 
wherein said input unit comprises a video camera. 

25. A face recognition system according to claim 1, 
wherein said input unit comprises a digital camera. 

26. A face recognition system according to claim 1, 
wherein said input unit comprises a memory comprising an 
image. 

27. A face recognition system according to claim 1, 
wherein the first facial feature comprises a single eye. 

28. A face recognition system according to claim 1, 
wherein the second facial feature comprises both eyes and 
nose. 

29. A face recognition system according to claim 1, 
wherein said face detector comprises a neural network. 

30. A face recognition system according to claim 1, 
wherein said face position registration circuit comprises a 
neural network. 

31. A face recognition system according to claim 1, 
wherein said feature extractor comprises a neural network. 

32. A face recognition system comprising: 

an input unit for inputting an image of a subject; 

a face position registration circuit for detennining a 
position of a face in the image from said input circuit, 
said face position registration circuit comprising an eye 
template circuit for applying an eye template to a first 
side of the face and a minor image of the eye template 
to a second side of the face; 

a feature extractor for extracting at least first and second 
facial features from the face; and 

a voting circuit for comparing the first and second facial 
features extracted by said feature extractor circuit with 
a database of extracted facial features to identify the 
face. 

33. A face recognition system according to claim 32, 
wherein said input unit comprises a video camera. 

34. A face recognition system according to claim 32, 
wherein said input unit comprises a digital camera. 
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35. A face recognition system according to claim 32, 
wherein said input unit comprises a memory comprising an 
image. 

36. A face recognition system according to claim 32, 
wherein the first facial feature comprises a single eye. 5 

37. A face recognition system according to claim 32, 
wherein the second facial feature comprises both eyes and 
nose. 

38. A face recognition system according to claim 32, 
wherein said face position registration circuit comprises a 10 
neural network. 

39. A face recognition system according to claim 32, 
wherein said feature extractor comprises a neural network. 

40. A method of face recognition comprising the steps of: 

(a) inputting an image of a subject; 15 

(b) determining if a face is present in the image output of 
step (a); 

(c) determining a position of the face in the image if step 
(b) determines that the face is present including apply- 
ing an eye template to a first side of the face and a 
mirror image of the eye template to a second side of the 
face; 

(d) extracting at least first and second facial features firom 
the face output by step (c); and 2S 

(e) comparing the first and second facial features output 
by step (d) to identify the face with a database of facial 
features. 

41. A method according to claim 40, wherein step (b) 
comprises the steps of: 30 

(bl) lumen normalizing the output of step (a); 

(b2) extracting edge information from an output step (bl); 

(b3) down sampling at a first predetermined ratio an 

output of step (bl); 
(b4) applying a plurality of facial templates to an output 3S 

of step (b3); and 
(b5) determining which one of the plurality of facial 

templates corresponds to the output of step (b4). 

42. A method according to claim 41, ^ 
wherein the output of step (b2) comprises MxN pixels, 

wherein M and N are positive integers, 
wherein step (b2) comprises the steps of: 
(b2I) low pass filtering the MxN pixels; 
(b2II) sampling every n* pixel from (b2I) to output 45 
M/nxM/n pixels, wherein n is a position integer. 

43. A method according to claim 40, wherein step (c) 
comprises the steps of: 

(cl) lumen normalizing an output of step (b); 

(c2) extracting edge information firom an output of step 50 

(cl); 

(c3) down sampling at a first predetermined ratio an 
output from step (c2); 

(c4) applying plurality of eye templates to an output step 
(c3); 55 

(c5) determining which one of the plurality of eye tem- 
plates corresponds to the output of step (c3); and 

(c6) aligning the face of the image inputted in step (a) in 
accordance with step (c5). 60 

44. A method of face recognition comprising the steps of: 

(a) inputting an image of a subject; 

(b) determining if a face is present in the image output of 
step (a); 

(c) determining a position of the face in the image if step 65 
(b) determines that the face is -present, wherein step (c) 
comprises the steps of: 
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(cl) lumen normalizing an output of step (b); 
(c2) extracting edge information from an output of step 
(cl): 

(c3) down sampling at a first predetermined ratio an 

output from step (c2): 
(c4) applying a plurality of eye templates to an output 

of step (c3); 

(c5) determining which one of the plurality of eye 
templates corresponds to the output of step (c3); and 

(c6) aligning the face of the image inputted in step (a) 
in accordance with step (c5); 

(d) extracting at least first and second facial features from 
the face output by step (c); and 

(e) comparing the first and second facial features output 
by step (d) to identify the face with a database of facial 
features; 

wherein step (c4) applies an eye template to a first side of 
the face and a mirror image of the eye template to a 
second side of the face. 

45. A method according to claim 43, wherein step (c4) 
applies an eye template comprising a right eye and a left eye. 

46. A method of face recognition comprising the steps of: 

(a) inputting an image of a subject; 

(b) detenmning if a face is present in the image output of 
step (a); 

(c) determining a position of the face in the image if step 
(b) determines that the face is present, wherein step (c) 
comprises the steps of: 
(cl) lumen normalizing an output of step (b); 
(c2) extracting edge information from an output of step 

(cl); 

(c3) down sampling at a first predetermined ratio an 

output from step (c2); 
(c4) applying a plurality of eye templates to an output 
of step (c3); 

(c5) determining which one of the plurality of eye 
templates corresponds to the output of step (c3); and 
(c6) aligning the face of the image inputted in step (a) 
in accordance with step (c5); 

(d) extracting at least first and second facial features from 
the face output by step (c); and 

(e) comparing the first and second facial features output 
by step (d) to identify the face with a database of facial 
features; 

wherein if step (c5) does not determine that any one of the 
plurality of eye templates corresponds to the output of 
step (c3), step (c3) down samples at a second prede- 
termined ratio the output signal from step (c2). 

47. A method of face recognition comprising the steps of: 

(a) inputting an image of a subject; 

(b) determining if a face is present in the image output of 
step (a); 

(c) determining a position of the face in the image if step 
(b) determines that the face is present; 

(d) extracting at least first and second facial features from 
the face output by step (c); and 

(e) comparing the first and second facial features output 
by step (d) to identify the face with a database of facial 
features; 

wherein step (d) comprises the steps of: 

(dl) cropping at least a first facial feature and a second 

facial feature output by step (c); 
(d2) lumen normalizing the first facial feature and the 
second facial feature output by step (dl); 
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(d3) extracting edge information from the first facial 
feature and the second facial feature output by step 

(«); 

(d4) down sampling at a first predetermined ratio the 
first facial feature and at a second predetermined 
ratio the second facial feature output signal by step 
(d3); and 

(d5) calculating a first feature vector of the first facial 
feature from step (d4) and a second feature vector of 
the second facial feature output from step (d4). 

48. A method according to claim 47, wherein the first 
predetermined ratio is greater than the second predetermined 
ratio. 

49. A method according to claim 47, further comprising 
the steps of 

(d6) cropping a third facial output by output by step (c); 
(d7) lumen normalizing the third facial feature output by 
step (d6); 

(d8) extracting edge information from the third facial 

feature output step (d7); 
(d9) down sampling at a third predetermined ratio the 

third facial feature output by step (d8); and 
(dlO)calculating a third feature vector of the third facial 

feature from (d9). 

50. A method according to claim 49, wherein the first 
predetermined ratio is greater than the second predetermined 
ratio, and wherein the second ratio is greater than the third 
ratio. 

51. A method of face recognition comprising the steps of: 

(a) inputting an image of a subject; 

(b) detenriining if a face is present in the image output of 
step (a); 

(c) determining a position of the face in the image if step 
(b) determines that the face is present; 

(d) extracting at least first and second facial features from 
the face output by step (c); and 

(e) comparing the first and second facial features output 
by step (d) to identify the face with a database of facial 
features; 

wherein step (d) comprises the steps of: 

(dl) cropping at least a first facial feature and a second 

facial feature output by step (c); 
(d2) down sampling at a first predetermined ratio the 

first facial feature and at a second predetermined 

ratio the second facial feature output signal by step 

(dl); and 

(d3) calculating a first feature vector of the first facial 
feature from step (d2) and a second feature vector of 
the second facial feature output from step (d2). 

52. A method according to claim 51, wherein the first 
predetermined ratio is greater than the second predetermined 
ratio. 

53. A method according to claim 51, further comprising 
the steps of: 

(d4) cropping a third facial output by step (c); 

(d5) down sampling at a third predetermined ratio the 

third facial feature output by step (d4); and 
(d6) calculating a third feature vector of the third facial 

feature from step (d5). 

54. A method according to claim 53, wherein the first 
predetermined ratio is greater than the second predetermined 
ratio, and wherein the second ratio is greater than the third 
ratio. 

55. A method according to claim 47, wherein step (e) 
comprises the steps of: 



(el) comparing the first feature vector calculated in step 
(5) with corresponding first feature vectors of a data- 
base of candidates; 

(e2) comparing the second feature vector calculated in 
5 step (5) with corresponding second features of the 
database of candidates; and 

(e3) selecting one of the candidates in the database in 
accordance with one step (el) and step (e2). 

56. A method according to claim 40, wherein step (e) 
10 comprises the steps of: 

(el) comparing the first feature extracted in step (d) with 
corresponding first features of a database of candidates; 

(e2) comparing the second feature extracted in step (d) 
15 with corresponding second features of the database of 
candidates; and 

(e3) selecting one of the candidates in the database in 
accordance with one step (el) and step (e2). 

57. A method according to claim 40, wherein step (b) 
20 comprises a neural network. 

58. A method according to claim 40, wherein step (c) 
comprises a neural network. 

59. A method according to claim 40, wherein step (d) 
comprises a neural network. 

25 60. A method of face recognition comprising the steps of: 

(a) generating an image of a subject; 

(b) determining a position of a face in the image from step 
(a) including applying an eye template to a first side of 
the face and a mirror image of the eye template to a 

30 second side of the face; 

(c) extracting at least two facial features from the face 
determined in step (b); and 

(d) comparing the extracted facial features of step (c) with 
a database of extracted facial features to identify the 

35 face. 

61. A method according to claim 60, wherein step (b) 
comprises a neural network. 

62. A method according to claim 60, wherein step (c) 
comprises a neural network. 

40 63. A face recognition system comprising: 

input means for inputting an image of a subject; 

face detector means for determining if a face is present in 

the image output by said input means; 
face position registration means for determining a posi- 
tion of the face in the image if said face detector means 
determines that the face is present, said face position 
registration means comprising eye template means for 
applying an eye template to a first side of the face and 
a mirror image of the eye template to a second side of 
the face; 

feature extractor means for extracting at least first and 
second facial features from the face in accordance with 
the position determined by said face position re gist ra- 
55 tion means; and 

voting means for comparing the at least two extracted 
facial features with a database of corresponding facial 
features to identify the face. 

64. A face recognition system according to claim 63, 
60 wherein said face detector means comprises: 

Lumen normalization means for lumen normalizing the 
output by said input means; 

edge extraction means for extracting edge information 
from an output of said lumen normalization means; 
65 down sampling means for down sampling at a predeter- 
mined ratio an output signal from said edge extraction 
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facial template means for applying plurality of facial 
templates to an output of said down sampling means; 
and 

facial template matching means for determining which 
one of the plurality of facial templates corresponds to 
the output of said down sampling means. 

65. A face recognition system according to claim 64, 
wherein the image output by said input means comprises 

MxN pixels, wherein M and N are positive integers, 
wherein said down sampling means comprises: 
low pass filtering means to filter the MxN pixels; 
sampler means for sampling every nth pixel from said 

low pass filter to output M/nxN/n pixels, wherein n 

is a positive integer. 

66. A face recognition system according to claim 63, 
wherein said face position registration means comprises: 

Lumen normalization means for lumen normalizing an 
output by said face detector means; 

edge extraction means for extracting edge information 
from an output of said lumen normalization means; 

down sampling means for down sampling at a first 
predetermined ratio an output signal from said edge 
extraction means; 

eye template means for applying a plurality of eye tem- 
plates to an output of said down sampling means; 

eye template matching means for determining which one 
of the plurality of eye templates corresponds to the 
output of said down sampling means; and 

face alignment means for aligning the face of the image 
output by said input means in accordance with said eye 
matching means. 

67. A face recognition system comprising: 
input means for inputting an image of a subject; 

face detector means for determining if a face is present in 

the image output by said input means; 
face position registration means for determining a posi- 
tion of the face in the image if said face detector means 
determines that the face is present, said face position 
registration means comprising: 
lumen normalization means for lumen normalizing an 

output by said face detector means; 
edge extraction means for extracting edge information 

from an output of said lumen normalization means; 
down sampling means for down sampling at a first 

predetermined ratio an output signal from said edge 

extraction means; 
eye template means for applying a plurality of eye 

templates to an output of said down sampling means; 
eye template matching means for determining which 

one of the plurality of eye templates corresponds to 

the output of said down sampling means; and 
face alignment means for aligning the face of the image 

output by said input means in accordance with said 

eye matching means; 
feature extractor means for extracting at least first and 
second facial features from the face in accordance with 
the position determined by said face position registra- 
tion means; and 
voting means for comparing the at least two extracted 
facial features with a database of corresponding facial 
features to identify the face; 
wherein said eye template means applies an eye template 
to a first side of the face and a mirror image of the eye 
template to a second side of the face. 
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68. A face recognition system according to claim 66, 
wherein said eye template means applies an eye template 
comprising a right eye and a left eye. 

69. A face recognition system comprising: 
input means for inputting an image of a subject; 

face detector means for determining if a face is present in 
the image output by said input means; 

face position registration means for determining a posi- 
tion of the face in the image if said face detector means 
determines that the face is present, said face position 
registration means comprising: 

lumen normalization means for lumen normalizing an 

output by said face detector means; 
edge extraction means for extracting edge information 

from an output of said lumen normalization means; 
down sampling means for down sampling at a first 

predetermined ratio an output signal from said edge 

extraction means; 
eye template means for applying a plurality of eye 

templates to an output of said down sampling means; 
eye template matching means for determining which 

one of the plurality of eye templates corresponds to 

the output of said down sampling means; and 
face alignment means for aligning the face of the image 

output by said input means in accordance with said 

eye matching means; 
feature extractor means for extracting at least first and 
second facial features from the face in accordance with 
the position determined by said face position registra- 
tion means; and 
voting means for comparing the at least two extracted 
facial features with a database of corresponding facial 
features to identify the face; 
wherein if said eye matching means does not determine 
that any one of the plurality of eye templates corre- 
sponds to the output of said down sample means, said 
down sampling means down samples at a second 
predetermined ratio the output signal from said edge 
extraction means. 

70. A face recognition system comprising: 
input means for inputting an image of a subject; 

face detector means for determining if a face is present in 

the image output by said input means; 
face position registration means for determining a posi- 
tion of the face in the image if said face detector 
means-determines that the face is present; 
feature extractor means for extracting at least first and 
second facial features from the face in accordance with 
the position determined by said face position registra- 
tion means; and 
voting means for comparing the at least two extracted 
facial features with a database of corresponding facial 
features to identify the face; 
wherein said feature extraction means comprises: 

cropping means for cropping at least the first facial 
feature and the second facial feature output by said 
face position registration means; 
lumen normalization means for lumen normalizing the 
first facial feature and the second facial feature 
output by said cropping means; 
edge extraction means for extracting edge information 
from the first facial feature and the second facial 
feature output by said lumen normalization means; 
down sampling means for down sampling at a first 
predetermined ratio the first facial feature and at a 
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second predetermined ratio the second facial feature 
output signal by said edge extraction means; and 
feature vector calculating means for calculating a first 
feature vector of the first facial feature from said 
down sampling means and a second feature vector of 
the second facial feature output from said down 
sampling means. 

71. A face recognition system according to claim 70, 
wherein the first predetermined ratio is greater than the 
second predetermined ratio. 

72. A face recognition system according to claim 70, 
wherein said cropping means crops a third facial output 

by said face position registration means; 

wherein said lumen normalization means lumen normal- 
izes the third facial feature output by said cropping 
means detector means; 

wherein said edge extraction means extracts edge infor- 
mation from the third facial feature output by said 
lumen normalization means; 

wherein said down sampling means down samples at a 
third predetermined ratio the third facial feature output 
by said extraction means; and 

wherein said feature vector calculating means calculates a 
third feature vector of the third facial feature from said 
down sampling means. 

73. A face recognition system according to claim 72, 
wherein the first predetermined ratio is greater than the 
second predetermined ratio, and wherein the second ratio is 
greater than the third ratio. 

74. A face recognition system comprising: 
input means for inputting an image of a subject; 

face detector means for determining if a face is present in 

the image output by said input means; 
face position registration means for determining a posi- 
tion of the face in the image if said face detector means 
determines that the face is present; 
feature extractor means for extracting at least first and 
second facial features from the face in accordance with 
the position determined by said face position registra- 
tion means; and 
voting means for comparing the at least two extracted 
facial features with a database of corresponding facial 
features to identify the face; 
wherein said feature extraction means comprises: 

cropping means for cropping at least a first facial 
feature and a second facial feature output by said 
face position registration means; 
down sampling means for down sampling at a first 
predetermined ratio the first facial feature and at a 
second predetermined ratio the second facial feature 
output signal by said cropping means; and 
feature vector calculating means for calculating a first 
feature vector of the first facial feature from said 
down sampling means and a second feature vector of 
the second facial feature output from said down 
sampling means. 

75. A face recognition system according to claim 74, 
wherein the first predetermined ratio is greater than the 
second predetermined ratio. 

76. A face recognition system according to claim 74, 
wherein said cropping means crops a third facial output 

by said face position registration means; 
wherein said down sampling means down samples at a 
third predetermined ratio the third facial feature output 
by said cropping means; and 
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wherein said feature vector calculating means calculates a 
third feature vector of the third facial feature from said 
down sampling means. 

77. A face recognition system according to claim 74, 
wherein said voting means comprises 

first recognizer means for comparing the first feature 
vector with corresponding first feature vectors of a 
database of candidates; 

second recognizer means for comparing the second fea- 
ture vector with corresponding second feature vectors 
of the database of candidates; and 

selection means of selecting one of the candidates in the 
database in accordance with one of said first recognizer 
means and said second recognizer means. 

78. A face recognition system according to claim 76, 
wherein the first predetermined ratio is greater than the 
second predetermined ratio, and wherein the second ratio is 
greater than the third ratio. 

79. A face recognition system according to claim 70, 
wherein said voting means comprises 

first recognizer means for comparing the first feature with 
corresponding first features of a database of candidates; 

second recognizer means for comparing the second fea- 
ture with corresponding second features of the database 
of candidates; and 

selection means of selecting one of the candidates in the 
database in accordance with one of said first recognizer 
means and said second recognizer means. 

80. A face recognition system according to claim 63, 
wherein said feature extractor means comprises a neural 
network. 

81. A face recognition system comprising: 
input means for inputting an image of a subject; 

face position registration means for determining a posi- 
tion of a face in the image output from said input 
means, said face position registration means compris- 
ing eye template means for applying an eye template to 
a first side of the face and a mirror image of the eye 
template to a second side of the face; 

feature extractor means for extracting at least first and 
second facial features from the face; and 

voting means for comparing the first and second facial 
features extracted by said feature extractor means with 
a database of extracted facial features to identify the 
face. 

82. A face recognition system according to claim 81, 
wherein said face position registration means comprises a 
neural network. 

83. An article of manufacture, comprising a computer 
usable medium having readable code means embodied 
therein for face recognition, the computer readable program 
code means comprising: 

input computer readable program code means for input- 
ting an image of a subject; 

face detector computer readable program code means for 
determining if a face is present in the image from said 
input computer readable program code means; 

face position registration computer readable program 
code means for determining a position of the face in the 
image if said face detector computer readable program 
code means determines that the face is present, said 
face position registration computer readable program 
code means comprising computer readable program 
code means for applying an eye template to a first side 
of the face and a mirror image of the eye template to a 
second side of the face; 
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feature extractor computer readable program code means 
for extracting at least first and second facial features 
from the face in accordance with the position deter- 
mined by said face position registration computer read- 
able program code means; and 

voting computer readable program code means for com- 
paring the at least two extracted facial features with a 
database of corresponding facial features to identify the 
face. 

84. An article of manufacture according to claim 83, 
wherein said face detector computer readable program code 
mean comprises a neural network. 

85. An article of manufacture according to claim 83, 
wherein said face position registration computer readable 
program code mean comprises a neural network. 

86. An article of manufacture according to claim 83, 
wherein said feature extractor computer readable program 
code mean comprises a neural network. 

87. A face recognition system comprising: 

a control panel having an input circuit and a display; 

a video camera for inputting an image of a subject in 
response to said control panel; 

a face detector for determining if a face is present in the 
image output by said video camera, if the face is not 
present a message is displayed by said display; 
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a face position registration circuit for deteniiining a 
position of the face in the image if said face detector 
determines that the face is present, said face position 
registration circuit comprising an eye template circuit 
that applies an eye template to a first side of the face 
and a mirror image of the eye template to a second side 
of the face; 

a feature extractor for extracting at least first and second 
facial features from the face in accordance with the 
position determined by said face position registration 
circuit; 

a face sever comprising a database of corresponding 

features of potential candidates; and 
voting circuit for comparing the at least two extracted 

facial features with the database from said face server 

to identify the face. 

88. A face recognition system according to claim 87, 
wherein said face detector comprises a neural network. 

89. A face recognition system according to claim 87, 
wherein said face position registration circuit comprises a 
neural network. 

90. A face recognition system according to claim 87, 
wherein said feature extractor comprises a neural network. 
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